Image processing device, autostereoscopic display device, and image processing method for parallax correction

ABSTRACT

According to an image processing device includes an acquiring unit and a correcting unit. The acquiring unit is configured to acquire a stereoscopic image containing a plurality of parallax images each having a mutually different parallax. The correcting unit is configured to perform, for each pixel of the stereoscopic image, correction to set a parallax number of a parallax image viewed at a viewpoint position to a first parallax number of a parallax image to be viewed at the viewpoint position by using correction data representing a difference value between the first parallax number and a second parallax number of a parallax image that is actually viewed at the viewpoint position and on which distortion correction for correcting distortion of light beams has been performed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2012-077612, filed on Mar. 29, 2012; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an image processingdevice, an autostereoscopic display device, an image processing methodand a computer program product.

BACKGROUND

There are image display devices allowing viewers to view stereoscopicimages. An image display device includes, on a front face of a displaypanel on which a plurality of pixels are arranged, a light beamcontroller that controls the emitting directions of light beams from thepixels, and displays a plurality of parallax images each having amutually different parallax.

For example, in a case of a sheet display or the like where a displayarea itself can be bent or in a case where parts of a panel and a lensare deformed or removed because of aged deterioration, there will occurregions where a stereoscopic image can be viewed and regions(pseudoscopic regions) where a stereoscopic image cannot be viewed inthe display area (screen), and viewers therefore cannot view astereoscopic image on the whole screen.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of an autostereoscopicdisplay device according to a first embodiment;

FIG. 2 is a diagram for explaining an example of pixel mapping;

FIG. 3 is a diagram for explaining an example of pixel mapping;

FIG. 4 is a diagram for explaining light reaching an eye at a certainviewpoint;

FIGS. 5A and 5B are schematic diagrams in a case where light beamdirections are uniform;

FIGS. 6A and 6B are schematic diagrams in a case where the light beamdirections are non-uniform;

FIG. 7 is a diagram illustrating an example of unit a plurality of unitregions into which a display area is divided;

FIG. 8 is a diagram illustrating an exemplary configuration of an imageprocessing unit according to the first embodiment;

FIG. 9 is a conceptual diagram of a correction process performed by acorrecting unit;

FIG. 10 is a flowchart illustrating an example of a process performed bythe image processing unit;

FIG. 11 is a diagram illustrating an exemplary configuration of an imageprocessing unit according to a second embodiment; and

FIG. 12 is a diagram for explaining an example of calculation ofcorrection data according to a position of a viewer.

DETAILED DESCRIPTION

According to an image processing device includes an acquiring unit and acorrecting unit. The acquiring unit is configured to acquire astereoscopic image containing a plurality of parallax images each havinga mutually different parallax. The correcting unit is configured toperform, for each pixel of the stereoscopic image, correction to set aparallax number of a parallax image viewed at a viewpoint position to afirst parallax number of a parallax image to be viewed at the viewpointposition by using correction data representing a difference valuebetween the first parallax number and a second parallax number of aparallax image that is actually viewed at the viewpoint position and onwhich distortion correction for correcting distortion of light beams hasbeen performed.

Various embodiments will be described below in detail with reference tothe accompanying drawings. An autostereoscopic display device accordingto the embodiments described below allows a viewer to view thestereoscopic images by displaying a plurality of parallax images eachhaving a mutually different parallax. The autostereoscopic displaydevice may employ a 3D display method such as the integral imagingmethod (II method) or the multi-viewpoint method. Examples of theautostereoscopic display device include a TV set, a PC, a smart phoneand a digital photo frame that enables the viewer to view stereoscopicimages with naked eyes.

First Embodiment

FIG. 1 is a schematic view of an autostereoscopic display device 1according to the first embodiment. The autostereoscopic display device 1includes a display unit 10 and an image processing unit 20.

The display unit 10 is a device capable of displaying the stereoscopicimages containing a plurality of parallax images each having a mutuallydifferent parallax. As illustrated in FIG. 1, the display unit 10includes a display panel 11 and a light beam control element 12.

A parallax image is an image used to allow a viewer to view astereoscopic image and refers to each of images constituting astereoscopic image. A stereoscopic image is an image in which the pixelsof the parallax images are assigned in such a way that, when the displaypanel 11 is viewed through the light beam control element 12 from aviewpoint position of a viewer, one of the parallax images is seen byone eye of the viewer and another parallax image is seen by the othereye of the viewer. Thus, a stereoscopic image is generated byrearranging the pixels of each parallax image.

The display panel 11 is a liquid crystal panel in which a plurality ofsub-pixels having color components (such as R, G, and B) are arranged ina matrix in a first direction (row direction) and a second direction(column direction). Alternatively, the display panel 11 may also be aflat panel such as an organic EL panel or a plasma panel. The displaypanel 11 illustrated in FIG. 1 is assumed to include a light source suchas a backlight. In the example of FIG. 1, one pixel is constituted bysub-pixels of R, G, and B. The sub-pixels are arranged such that apattern in an order of R (red), G (green) and B (blue) is repeated inthe first direction and the same color components are arranged in thesecond direction.

The light beam control element 12 controls the outgoing direction oflight beam coming out from each sub-pixel on the display panel 11. Thelight beam control element 12 has a plurality of optical openings, eachextending in a linear fashion and each allowing a light beam to go outtherethrough, is arranged along the first direction. In the example ofFIG. 1, the light beam control element 12 is lenticular sheet having aplurality of cylindrical lenses (serving as the optical openings)arranged thereon, but the light beam control element 12 is not limitedthereto and may also be a parallax barrier having a plurality of slitsarranged thereon, for example. Meanwhile, the display panel 11 and thelight beam control element 12 have a certain distance (gap) maintainedtherebetween. In addition, the light beam control element 12 is arrangedin such a way that the extending direction of the optical openings has apredetermined tilt with respect to the second direction (columndirection) of the display panel 11, and the positions of the opticalopenings and display pixels in the row direction are therefore shifted,as a result of which the viewing zone (the area where a stereoscopicimage can be viewed) therefore varies depending on the height.

In the first embodiment, since lenses are arranged obliquely on thepixels as illustrated in FIG. 2, pixels viewed through the lenses arethose along dotted lines in FIG. 2, for example. Specifically, since thepixels within the display panel 11 are arranged in a matrix along thehorizontal direction and the vertical direction while the lenses arearranged obliquely, the pixels need to be assigned along the extendingdirection of the lenses in assignment (pixel mapping) of pixels todisplay parallax images. A case of assignment of pixels each displayingone of seven parallax images will be described here as an example.Pixels having the same number are pixels displaying the same parallaximage. Parallax images are assigned with numbers (parallax numbers) thatare different from one another and different by 1 from those of adjacentparallax images in advance. A parallax number v of a pixel (i, j) on ai-th row and a j-th column on which pixel mapping is performed among aplurality of pixels arranged on the display panel 11 can be obtained bythe following expression (1), for example.

$\begin{matrix}{{v\left( {i,j} \right)} = {\frac{{mod}\left( {\left( {i + i_{offset} - {3{j \cdot a}\;\tan}} \right),X} \right)}{X_{n}}N}} & (1)\end{matrix}$

In the expression (1), i_(offset) represents a phase shift between animage and a lens and is represented in units of pixels. In the exampleof FIG. 3, an upper-left end of an image is a reference point (point oforigin) and a shift amount between the reference point and an upper-leftend of the lens is represented by i_(offset).

The parallax numbers v are continuous values but parallax images arediscrete, and therefore a parallax image itself cannot be assigned to v.Therefore, interpolation such as linear interpolation or cubicinterpolation is used. In this manner, a plurality of parallax imageseach having a mutually different parallax is displayed on the displayunit 10.

Next, light reaching an eye at a certain viewpoint will be described.FIG. 4 is a diagram schematically illustrating a light beam distributionin a case where the number of lenses 201 is one, the number of parallaximages (the number of parallaxes) is three and the width and the heightof each parallax image are both 1 (which can be regarded as a pixel inthis case). A light beam coming out from one pixel enters the lens 201and is ejected in a direction as controlled. Since light beams arediffused, the intensity of the light beams measured at positions at acertain distance from the lens 201 is distributed within a certain rangeas illustrated in FIG. 4. Note that the horizontal axis represents theposition in the first direction, the vertical axis represents theintensity of light, the light beam distribution of a pixel 501 isdenoted by 506, the light beam distribution of a pixel 502 is denoted by505, and the light beam distribution of a pixel 503 is denoted by 504.

Light reaching an eye when the lens 201 is viewed from a certainviewpoint is light in which pixel values of the pixels are overlaid(colors are mixed) according to the light beam distributions (504, 505,and 506). For example, light reaching an eye when the lens 201 is viewedfrom a position 507 corresponding to a certain viewpoint is representedby pixel values of the pixels 501, 502 and 503 weighted with values 508,509 and 510 at the position 507 of the light beam distributions asweights. Similarly, in a case of a plurality of lenses, a viewed imagethat can be viewed when an autostereoscopic display device of the IImethod is viewed with an eye from a viewpoint is an overlay image inwhich pixels of a plurality of parallax images are overlaid according tothe intensities of the light beam distributions at the viewpoint.Parallax images are images in which the position of an object is shiftedfrom one another so as to produce an effect of binocular disparity.Since the light beam distribution varies according to the viewpoint, theviewed image is an image varying according to the viewpoint. Although acase where the number of parallaxes is three is described in thisexample, the number of parallaxes may be designed to be any number. Forexample, the number of parallaxes may be nine. The description of anembodiment in such case will be similar to that in the case where thenumber of parallaxes is three.

FIGS. 5A and 5B are diagrams illustrating a case where the directions oflight beams coming out from pixels with the same parallax number areuniform when the number of parallaxes is nine. Uniform light beamdirections mean that the directions of light beams coming out frompixels with the same parallax number correspond to a predetermineddirection. More specifically, uniform light beam directions mean thatpixels of parallax images identified by the same parallax number areviewed at a certain viewpoint position (one eye).

FIG. 5A is a diagram illustrating parallax numbers of parallax imagesviewed in a display area of a stereoscopic image on the display unit 10from a certain viewpoint position in a case where the light beamdirections are uniform. In this example, it is assumed that parallaximages to be viewed from the viewpoint position m are those with theparallax number 5. In this case, since light coming out from the pixelsof the parallax images with the parallax number 5 enters the viewpointposition m from the display unit 10 as illustrated in FIG. 5B, theviewer can view parallax images with the parallax number 5 over all theregions of the display unit 10 when the viewer views the display unit 10from the viewpoint position m with one eye.

FIGS. 6A and 6B are diagrams illustrating a case where the directions oflight beams coming out from pixels with the same parallax number arenon-uniform when the number of parallaxes is nine. Non-uniform lightbeam directions mean that the directions of light beams coming out frompixels with the same parallax number include directions different fromthe predetermined direction. More specifically, non-uniform light beamdirections mean that pixels of parallax images identified by differentparallax numbers are viewed at a certain viewpoint position (one eye).

FIG. 6A is a diagram illustrating parallax numbers of parallax imagesviewed in the display area on the display unit 10 from a certainviewpoint position in a case where the light beam directions arenon-uniform. In this example, it is assumed that parallax images to beviewed from the viewpoint position m are those with the parallax number5. In this case, since light coming out from the pixels of the parallaximages with different parallax numbers (1, 4, 5 and 9) enters theviewpoint position m from the display unit 10 as illustrated in FIG. 6B,the viewer will view parallax images with different parallax numberswhen the viewer views the display unit 10 from the viewpoint position mwith one eye. Thus, when the light beam directions are non-uniform,there will occur regions where a stereoscopic image can be viewed andregions where a stereoscopic image cannot be viewed (pseudoscopicregions) in the display area (screen), and the viewer therefore cannotview a stereoscopic image on the whole screen.

As described above, there may be cases where the directions of lightbeams coming out from the pixels of the parallax images are differentfrom expected directions owing to distortion of the display unit 10,partial separation of the panel and the lens due to aged deteriorationor the like, for example. In such case, parallax images with parallaxnumbers that are different from those of parallax images to be viewedfrom a certain viewpoint position are viewed, the resulting entirescreen is not a normal viewing zone, and thus a stereoscopic visioncannot be obtained. In other words, there will occur regions where astereoscopic image can be viewed and regions where a stereoscopic imagecannot be viewed in the display area of the display unit 10.

Therefore, in the first embodiment, the image processing unit 20performs correction to set a parallax number of a parallax image viewedat a certain viewpoint position to a first parallax number of a parallaximage to be viewed at the viewpoint position for each pixel of astereoscopic image by using correction data representing a differencevalue between the first parallax number and a second parallax number ofa parallax image that is actually viewed at the viewpoint position. Theimage processing unit 20 then displays the corrected stereoscopic image(a plurality of parallax image; parallax image group) on the displayunit 10. As a result, the autostereoscopic display device 1 of the firstembodiment can make the screen a normal viewing zone. Specificdescription will be given below.

Before description of the image processing unit 20, the correction datamentioned above will be described. While an example in which theparallax number is nine will be described here, cases of other numbersof parallaxes can also be considered similarly. In the followingdescription, it is assumed that a parallax number “0” is assigned to afirst parallax image, a parallax number “1” is assigned to a secondparallax image adjacent thereto, a parallax number “2” is assigned to athird parallax image adjacent thereto, a parallax number “3” is assignedto a fourth parallax image adjacent thereto, a parallax number “4” isassigned to a fifth parallax image adjacent thereto, a parallax number“5” is assigned to a sixth parallax image adjacent thereto, a parallaxnumber “6” is assigned to a seventh parallax image adjacent thereto, aparallax number “7” is assigned to an eighth parallax image adjacentthereto, and a parallax number “8” is assigned to a second parallaximage adjacent thereto.

In addition, the display area of the display unit 10 is divided into aplurality of unit regions P arranged in a matrix of M rows and N columnshere as illustrated in FIG. 7. For example, a unit region P on an m-throw (≦M) and an n-th column (≦N) is expressed as P(m, n). Each unitregion P is a region in which at least one element image containingpixels with respective parallax numbers 0 to 8 is displayed. In thisexample, description will be made assuming that one unit region P is aregion in which one element image is displayed.

In the first embodiment, correction data representing a difference valuebetween the first parallax number to be viewed at a certain viewpointposition and the second parallax number that is actually viewed at theviewpoint among parallax images with parallax numbers 0 to 8 displayedin a unit region P is obtained for each unit region P, and the obtainedcorrected data and the unit region P are stored in a storage unit inassociation with each other. The viewpoint position for calculating thecorrection data can be set to any position, but an example in which theviewpoint at which a parallax image with the parallax number 4 is to beviewed is set to the viewpoint position for calculating the correctiondata will be described in this example. In the following, the viewpointposition for calculating the correction data (the viewpoint position atwhich a parallax image with the parallax number 4 is to be viewpoint inthis example) will be referred to as an viewpoint position forcorrection.

In addition, data representing what parallax number is actually viewedat a certain viewpoint position (the viewpoint position for correction,for example) will be referred to as light beam direction data. Lightbeam direction data representing a parallax number that is actuallyviewed at the viewpoint position for correction among the parallaxnumbers 0 to 8 displayed in a unit region P(m, n) will be expressed asL(m, n). The light beam direction data can also be regarded ascorresponding to “a second parallax number” in claim 2. The method forobtaining the light beam direction data may be any method. For example,the light beam direction can be obtained by moving a luminance meter formeasuring the luminance of a light beam coming out from the display unit10, by visually checking parallax images that are sequentially lit up,or by imaging parallax images that are sequentially lit up (a pluralityof images may be imaged at a time) and analyzing the imaging results.

While an example of a method of calculating the correction data c(m, n)associated with the unit region P(m, n) is described, a method forcalculating the correction data associated with another unit region Pcan similarly be considered. The correction data associated with theunit region P(m, n) is a difference value between the first parallaxnumber k_(dst) (4 in this example) to be viewed at the viewpointposition for correction and the light beam direction data L(m, n)representing the parallax number that is actually viewed at theviewpoint position for correction among the parallax numbers 0 to 8displayed in the unit region P(m, n), and can be obtained by thefollowing expression (2).C(m,n)=k _(dst) −L(m,n)  (2)

For example, when the parallax number actually viewed at the viewpointposition for correction is 2 among the parallax numbers 0 to 8 displayedin the unit region P(m, n), the correction data c(m, n) will be “2(=4−2)”. In this manner, correction data are obtained for each unitregion P, and the obtained correction data and the unit regions P arestored in advance in the storage unit in association with each other. Onthe above assumption, specific details of the image processing unit 20according to the first embodiment will be described.

FIG. 8 is a block diagram illustrating an exemplary configuration of theimage processing unit 20. As illustrated in FIG. 8, the image processingunit 20 includes an acquiring unit 21, a storage unit 22, a correctingunit 23 and an output unit 24. The acquiring unit 21 acquires K (K is aninteger of 2 or larger) parallax images used for a stereoscopic image.In other words, the acquiring unit 21 has a function of acquiring astereoscopic image containing a plurality of parallax images.

The storage unit 22 stores therein the correction data described above.More specifically, the storage unit 22 stores each of a plurality ofunit regions P and the correction data in association with each other.

The correcting unit 23 corrects a parallax number of a sub-pixelcalculated by pixel mapping described above by using the correction datastored in the storage unit 22 for each sub-pixel of the display panel11. Specific description will be given below. Herein, the number ofsub-pixels in the first direction (row direction) of the display panel11 is represented by W, the number of sub-pixels in the second direction(column direction) is represented by H, and position coordinates of asub-pixel of the display panel 11 arranged below the lens arerepresented by (i, j)^(T). T represents a transpose.

The relation between the sub-pixel position (i, j)^(T) and thecoordinate position (m, n)^(T) of the unit region P is expressed by thefollowing expression (3).

$\begin{matrix}{\begin{bmatrix}m \\n\end{bmatrix} = \begin{bmatrix}{\frac{M}{W}i} \\{\frac{N}{H}j}\end{bmatrix}} & (3)\end{matrix}$

Accordingly, the correction data for the unit region P corresponding tothe sub-pixel position (i, j)^(T) are expressed by the followingexpression (4).

$\begin{matrix}{{{correction}\mspace{14mu}{data}\mspace{14mu}{for}\mspace{14mu}{sub}\text{-}{pixel}\mspace{14mu}{position}\mspace{14mu}\left( {i,j} \right)^{T}} = {C\left( {{\frac{M}{W}i},{\frac{N}{H}j}} \right)}} & (4)\end{matrix}$

Note that the coordinate position for the unit region P corresponding tothe sub-pixel position (i, j)^(T) may be a value that is not an integer.In this case, the correction data for the sub-pixel position (i, j)^(T)can be obtained by performing interpolation using adjacent correctiondata, for example.

The correcting unit 23 modifies the parallax number of each sub-pixelaccording to a value indicated by the correction data for the unitregion P corresponding to the sub-pixel. More specifically, thecorrecting unit 23 modifies the parallax number of each sub-pixel to aparallax number indicated by a remainder resulting from dividing a valueobtained by adding a value of the parallax number before correctioncalculated by the pixel mapping described above and a value ofcorrection data associated with the sub-pixel by the number ofparallaxes K. For example, when the parallax number before thecorrection of a sub-pixel (i, j) on an i-th row and a j-th column isrepresented by v(i, j), the parallax number v′(i, j) of the sub-pixel(i, j) after the correction by the correcting unit 23 can be expressedby the following expression (5).

$\begin{matrix}{{v^{\prime}\left( {i,j} \right)} = {{mod}\left( {{{v\left( {i,j} \right)} + {C\left( {{\frac{M}{W}i},{\frac{N}{H}j}} \right)}},K} \right)}} & (5)\end{matrix}$

The parallax image to be arranged at the position coordinates (i, j)^(T)can be expressed by the following expression (6).I′(i,j)=I _(v′(i,j))(i,j)  (6)

Note that the parallax number v′(i, j) may not be an integer but adecimal. In such case, the parallax image can be obtained by linearinterpolation as in the following expression (7).k ₀=mod(└v′(i,j)┘,K)k ₁=mod(└v′(i,j)┘+1,K)α=v′(i,j)−k ₀I′=(i,j)=(1−α)Ik ₀(i,j)+αIk ₁  (7)where └x┘ represents a maximum integer not larger than x, andmod(x, K) represents a remainder resulting from dividing x by K.

FIG. 9 is a conceptual diagram of a correction process performed on eachpixel with parallax numbers 0 to 8 corresponding to a certain unitregion P. In the example of FIG. 9, the parallax number to be viewed atthe viewpoint position for correction is “4” while the parallax numberactually viewed at the viewpoint position for correction among theparallax numbers 0 to 8 corresponding to a certain unit region P is “2”(that is, the light beam direction data of the unit region P=2).Accordingly, the correction data corresponding to the unit region P is“2” (=4−2), and each pixel corresponding to the unit region P ismodified to a pixel with the parallax number indicated by the remainderresulting from dividing a value obtained by adding “2” to the parallaxnumber before the correction by the parallax number “9” (see FIG. 9). Asdescribed above, the correcting unit 23 modifies the arrangement of thesub-pixels so that the parallax image with the parallax number 4 (theparallax image to be viewed at the viewpoint position for correction)will be viewed at the viewpoint position for correction. As a result,the parallax image with the parallax number “4” is viewed at theviewpoint position for correction. The correcting unit 23 output thecorrected parallax image group to the output unit 24. The output unit 24outputs the parallax image group received from the correcting unit 23(in other words, a stereoscopic image corrected by the correcting unit23) to the display panel 11 of the display unit 10.

FIG. 10 is a flowchart illustrating an example of a process performed bythe image processing unit 20 according to the first embodiment. Asillustrated in FIG. 10, the acquiring unit 21 first acquires a pluralityof parallax image (step S1). The acquiring unit 21 outputs the acquiredparallax image group to the correcting unit 23. The correcting unit 23specifies any of a plurality of sub-pixels to be arranged in a matrix onthe display panel 11 (step S2). The correcting unit 23 calculates theparallax number of the sub-pixel specified in step S2 by the expression(1) described above (step S3). The correcting unit 23 obtains correctiondata for the unit region P corresponding to the sub-pixel specified instep S2 and performs a correcting process on the sub-pixels (step S4).

If the correction process has been completed on all the sub-pixels (ifthe result of step S5 is YES), the correcting unit 23 outputs thecorrected parallax image group to the output unit 24. The output unit 24then displays the parallax image group received from the correcting unit23 on the display panel 11 (step S6) and terminates this routine. If thecorrection process has not been completed on all the sub-pixels (if theresult of step S5 is NO), on the other hand, the process returns to stepS2 described above where the correcting unit 23 sequentially specifies asub-pixel on which the correction process has not been completed.

As described above, according to the first embodiment, since correctionto set a parallax number of a parallax image viewed at the viewpointposition for correction to a first parallax number of a parallax imageto be viewed at the viewpoint position for correction is performed foreach pixel of a stereoscopic image by using correction data representinga difference value between the first parallax number and a secondparallax number of a parallax image that is actually viewed at theviewpoint position for correction, it is possible to suppress occurrenceof regions where a stereoscopic image can be viewed and regions(pseudoscopic regions) where a stereoscopic image cannot be viewed in adisplay area (screen) of a stereoscopic image. In other words, accordingto the first embodiment, it is possible to make a screen a normalviewing zone.

Modified Example of First Embodiment

The obtained light beam direction data, however, are likely to bedistorted into a trapezoid or the like instead of a rectangular that isthe panel shape owing to distortion of a camera lens or the like.Coordinate transformation for correction of such distortion can beexpressed by the following expression (8), for example.

$\begin{matrix}{\begin{bmatrix}{\delta_{1}\left( {m,n} \right)} \\{\delta_{2}\left( {m,n} \right)}\end{bmatrix} = {\delta\left( \begin{bmatrix}m \\n\end{bmatrix} \right)}} & (8)\end{matrix}$

When projective transformation is used, for example, the coordinatetransformation can be expressed by the following expression (9).

$\begin{matrix}{{\delta\left( \begin{bmatrix}m \\n\end{bmatrix} \right)} = \begin{bmatrix}\frac{{a_{0}m} + {a_{1}n} + a_{2}}{{a_{6}m} + {a_{7}n} + 1} \\\frac{{a_{3}m} + {a_{4}n} + a_{5}}{{a_{8}m} + {a_{9}n} + 1}\end{bmatrix}} & (9)\end{matrix}$

In the expression (9), a₀ to a₉ represent parameters for projectivetransformation that can be obtained from coordinates of four corners ofthe panel in the obtained light beam direction data and four corners ofa liquid crystal panel (display panel 11). Accordingly, the correctiondata for the sub-pixel position (i, j)^(T) are expressed by thefollowing expression (10).

$\begin{matrix}{{{correction}\mspace{14mu}{data}\mspace{14mu}{for}\mspace{14mu}{sub}\text{-}{pixel}\mspace{14mu}{position}\mspace{11mu}\left( {i,j} \right)^{T}} = {C\left( {{\delta_{1}\left( {{\frac{M}{W}i},{\frac{N}{H}j}} \right)},{\delta_{2}\left( {{\frac{M}{W}i},{\frac{N}{H}j}} \right)}} \right)}} & (10)\end{matrix}$

Thus, the correction data for a sub-pixel can also be regarded as beingexpressed by a difference value between a first parallax number to beviewed at the viewpoint position for correction and a second parallaxnumber that is actually viewed at the viewpoint position for correctionamong a plurality of parallax numbers displayed on the unit region Pcorresponding to the sub-pixel and on which distortion correction forcorrecting distortion of the light beam is performed.

Note that the coordinate position for the unit region P corresponding tothe sub-pixel position (i, j)^(T) may be a value that is not an integer.In this case, the correction data for the sub-pixel position (i, j)^(T)can be obtained by performing interpolation using correction data foradjacent sub-pixel positions, for example.

Similarly to the first embodiment described above, the correcting unit23 modifies the parallax number of each sub-pixel according to a valueindicated by the correction data for the sub-pixel. More specifically,the correcting unit 23 modifies the parallax number of each sub-pixel toa parallax number indicated by a remainder resulting from dividing avalue obtained by adding a value of the parallax number beforecorrection calculated by the pixel mapping described above and a valueof correction data for the sub-pixel by the number of parallaxes K. Forexample, when the parallax number before the correction of a sub-pixel(i, j) on an i-th row and a j-th column is represented by v(i, j), theparallax number v′(i, j) of the sub-pixel (i, j) after the correction bythe correcting unit 23 can be expressed by the following expression(11).

$\begin{matrix}{{v^{\prime}\left( {i,j} \right)} = {{mod}\left( {{{v\left( {i,j} \right)} + {C\left( {{\delta_{1}\left( {{\frac{M}{W}i},{\frac{N}{H}j}} \right)},{\delta_{2}\left( {{\frac{M}{W}i},{\frac{N}{H}j}} \right)}} \right)}},K} \right)}} & (11)\end{matrix}$

Since other details are similar to those in the first embodiment,detailed description thereof will not be repeated.

Second Embodiment

In the first embodiment described above, the aforementioned correctionis performed by using the correction data at one viewpoint position (theviewpoint position for correction). In contrast, the second embodimentdiffers from the first embodiment in that a storage unit storingcorrection data at each of a plurality of assumed viewpoint positions(referred to as “candidate viewpoint positions”) are stored in advanceis provided and in that the correction data associated with a positionof the viewer are obtained from the position of the viewer andinformation stored in the storage unit and the aforementioned correctionis performed by using the obtained correction data. Specific descriptionwill be given below. Components designated by the same referencenumerals as in the first embodiment have similar functions and redundantdescription thereof will not be repeated as appropriate.

FIG. 11 is a diagram illustrating an exemplary configuration of an imageprocessing unit 200 according to the second embodiment. As illustratedin FIG. 11, the image processing unit 200 includes an acquiring unit 21,a storage unit 220, a detecting unit 25, a correcting unit 230 and anoutput unit 24. The storage unit 220 stores therein candidate positioninformation representing candidate viewpoint positions and correctiondata for each unit region P in association with each other for aplurality of candidate viewpoint positions. The correction data for eachunit regions P at each candidate viewpoint position are calculated inadvance and stored in the storage unit 220 similarly to the firstembodiment. The number of candidate viewpoint positions can be set toany value according to design conditions or the like.

The detecting unit 25 detects the position of the viewer. Morespecifically, the detecting unit 25 acquires a picture from a camera anddetects a region of faces of one or more viewers by means of a knownface detection method. The detecting unit 25 then sets positions of anyof a right eye, a left eye and both eyes and determines spatial X and Ycoordinates of the viewer. For example, a position where an eye existswith highest probability in a face region may be set as the position ofan eye of the viewer. Alternatively, the positions of a right eye, aleft eye and both eyes can also be detected by means of a known eyedetection method from an image captured by a camera. Alternatively, itis also possible to estimate a distance to the viewer by using the sizeof a face on an image captured by a camera and determine a spatial Zcoordinate of the viewer. The detecting unit 25 detects one or more ofthese X, Y and Z coordinates as viewer position information representingthe position of the viewer and outputs the detected viewer positioninformation to the correcting unit 230.

When the viewer position information detected by the detecting unit 25is coincident with any of a plurality of pieces of candidate viewpointposition information stored in the storage unit 220, the correcting unit230 performs the aforementioned correction by using the correction dataassociated with the candidate viewpoint position information that iscoincident with the viewer position information detected by thedetecting unit 25.

When the viewer position information detected by the detecting unit 25is not coincident with any of the pieces of candidate viewpoint positioninformation, on the other hand, the correcting unit 230 performsinterpolation to obtain correction data associated with the viewerposition information. As illustrated in FIG. 12, for example, when thereare six pieces of candidate viewpoint positions S, i.e., S₁, S₂, S₃, S₄,S₅, and S₆, set in advance and the position Vx of the viewer representedby the viewer position information detected by the detecting unit 25 isnot coincident with any of the candidate viewpoint positions S, thecorrecting unit 230 determines a candidate viewpoint position S with adistance to the positions Vx of the viewer being a threshold or shorter.In the example of FIG. 12, the candidate point positions S₁, S₂, S₄ andS₅ are determined. The correcting unit 230 then obtains the correctiondata associated with the position Vx of the viewer by performing linearinterpolation according to a distance α between the candidate viewpointposition S₁ and the position Vx of the viewer, a distance β between thecandidate viewpoint position S₂ and the position Vx of the viewer, adistance γ between the candidate viewpoint position S₄ and the positionVx of the viewer, and a distance δ between the candidate viewpointposition S₅ and the position Vx of the viewer. The correcting unit 230then performs the aforementioned correction by using the obtainedcorrection data.

In the embodiments described above, the light beam control element 12 isarranged in such a way that the extending direction of the opticalopenings has a predetermined tilt with respect to the second direction(column direction) of the display panel 11, and an angle or the tilt canbe arbitrarily changed. Alternatively, the light beam control element 12may have a structure (a so-called vertical lens) in which the extendingdirection of the optical openings thereof is coincident with the seconddirection of the display panel 11, for example.

In the first embodiment described above, for example, the storage unit22 may be configured to store the light beam direction data for eachunit region P and the parallax number to be viewed at the viewpointposition for correction instead of the correction data. In other words,the correcting unit 23 can obtain the light beam direction dataassociated with a sub-pixel to be corrected and the parallax number tobe viewed at the viewpoint position for correction from the informationstored in the storage unit 22, then calculate the correction data thatare a difference value between the light beam direction data and theparallax number, and perform the aforementioned correction by using thecalculated correction data. In other words, the correcting unit 23 maybe configured to generate the correction data. The light beam directiondata for each unit region P and the parallax number to be viewed at theviewpoint position for correction may be stored in separate storageunits.

Similarly, in the second embodiment described above, the storage unit220 may be configured to store the light beam direction data for eachcandidate viewpoint position and the parallax number to be viewed ateach candidate viewpoint position instead of storing the correction datafor each candidate viewpoint position (that is, the correcting unit 230may be configured to generate correction data for each candidateviewpoint position).

The image processing unit (20, 200) in the embodiments described aboveis a hardware configuration including a CPU (central processing unit), aROM, a RAM, a communication I/F unit, etc. The functions of respectivecomponents described above are implemented by expanding and executingvarious programs stored in the ROM on the RAM. Alternatively, at leastpart of the functions of the components can be implemented by anindividual circuit (hardware).

Alternatively, the programs to be executed by the image processing unit(20, 200) according to the embodiments described above may be stored ona computer system connected to a network such as the Internet, andprovided by being downloaded via the network. Alternatively, theprograms to be executed by the image processing unit (20, 200) accordingto the embodiments described above may be provided or distributedthrough a network such as the Internet. Still alternatively, theprograms to be executed by the image processing unit (20, 200) in theembodiments described above may be embedded on a ROM or the like inadvance and provided therefrom.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An image processing device, comprising: anacquiring unit configured to acquire a stereoscopic image containing aplurality of parallax images each having a mutually different parallax;and a correcting unit configured to perform, for each pixel of thestereoscopic image, correction to set a parallax number of a parallaximage viewed at a viewpoint position to a first parallax number of aparallax image to be viewed at the viewpoint position by usingcorrection data representing a difference value between the firstparallax number and a second parallax number of a parallax image that isactually viewed at the viewpoint position and on which distortioncorrection for correcting distortion of light beams has been performed,wherein the correcting unit modifies, for each pixel of the stereoscopicimage, the parallax number of the pixel to a parallax number indicatedby a remainder value resulting from dividing a value obtained by addinga value of the parallax number before correction and a value of thecorrection data for the pixel by the number of parallaxes.
 2. The deviceaccording to claim 1, wherein a display area of a display unit capableof displaying the stereoscopic image is divided in advance into aplurality of unit regions, each of the unit regions being a region inwhich an element image containing pixels of the parallax images isdisplayed, and the correcting unit modifies the parallax number of eachpixel of the stereoscopic image according to a value indicated by thecorrection data for the unit region corresponding to the pixel.
 3. Thedevice according to claim 1, further comprising: a storage unit thatstores therein each of a plurality of viewpoint positions in associationwith the correction data; and a detecting unit that detects a positionof a viewer, wherein the correcting unit obtains the correction dataassociated with the position of the viewer detected by the detectingunit from the position of the viewer and information stored in thestorage unit, and performs the correction by using the obtainedcorrection data.
 4. An autostereoscopic display device, comprising: anacquiring unit configured to acquire a stereoscopic image containing aplurality of parallax images each having a mutually different parallax;a correcting unit configured to perform correction to set a parallaxnumber of a parallax image viewed at a viewpoint position to a firstparallax number of a parallax image to be viewed at the viewpointposition for each pixel of the stereoscopic image by using correctiondata representing a difference value between the first parallax numberand a second parallax number of a parallax image that is actually viewedat the viewpoint position and on which distortion correction forcorrecting distortion of light beams has been performed; and a displayunit that displays the stereoscopic image on which the correction by thecorrecting unit is performed, wherein the correcting unit modifies, foreach pixel of the stereoscopic image, the parallax number of the pixelto a parallax number indicated by a remainder value resulting fromdividing a value obtained by adding a value of the parallax numberbefore correction and a value of the correction data for the pixel bythe number of parallaxes.
 5. An image processing method, comprising:acquiring a stereoscopic image containing a plurality of parallax imageseach having a mutually different parallax; and performing, for eachpixel of the stereoscopic image, correction to set a parallax number ofa parallax image viewed at a viewpoint position to a first parallaxnumber of a parallax image to be viewed at the viewpoint position byusing correction data representing a difference value between the firstparallax number and a second parallax number of a parallax image that isactually viewed at the viewpoint position and on which distortioncorrection for correcting distortion of light beams has been performed,wherein the correction includes modifying, for each pixel of thestereoscopic image, the parallax number of the pixel to a parallaxnumber indicated by a remainder value resulting from dividing a valueobtained by adding a value of the parallax number before correction anda value of the correction data for the pixel by the number ofparallaxes.